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X'^A PREFACE 



This report was prepared as part of Rand's DoD Training and Man-- 
power Management Program, sponsored by the Cybernetics Technology 
Office of the Defense Advanced Research Projects Agency (ARPA) . With 
manpower issues assuming an ever greater importance in defense plan- 
ning and budgeting, the purpose of this research program is' to develop 
broad strategies and specific solutions for dealing with present and 
future mililiary manpower problems. This includes the development of 
new research methodologies for examining broad classes of manpower 
problems, as well as specific problem-oriented research. In addition 
to providing analysis of current and future manpcwer issues, it is 
hoped that this research program will contribute to a better general 
understanding of the manpower problems confronting the Department of 
Defense. ^ • , 

Military training ftas received substantial attention in recent 
years as one area in which improved efficiency can lead to substan- 
tial cost reductions. This report treats an aspect of training effi- 
ciency that has to date received very little attention — the question 
of the efficient amount of formal training. Specifically, it deals 
with the efficient mix between formal and on-the-job training in pro- 
viding first-term enlisted personnel with the skills needed to perform 
effectively in their 'military specialties. Since foYmal initial spe- 
cialty training for flrst-teritl enlisted personnel currently costs 
about $2 billion per year, increased efficiency in this area could 
lead to substantial, recurring savings. This report describes problems 
involved in analyzing tradeoffs between formal and on-the-job training, 
alternative solutions to these problems, and some early results of Rand 
research in this area. . ' 

The substance af this work was originally presented at the Rand 

4 

Conference on Defense Manpower, February 4-6, 1976, and Is Included 
in the conference proceedings: Defenee' Manpower Policy: PveAumtations 
from the. 1976 Rand Conference on Defense Manpower^ edited by Richard 
V* L. Coop^, R-2396-ARPA, December 1978. Th^ present report is being 
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publlHhed separately biuaiise, an i)ne ot* several plaixhed |)ul) llcallonH 
dealliiK with Hand's training and prodiutlvlty roHearoh, It Is expected 
ti) have an ^Md^em e dil t erent from that tDr the con£er«nce proceed Ings 
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SUMMARY 



The U.S. military Is, among other things, a very large training 
Institution. It Is estimated that the cost of formal military train- 
ing during FY 1976 was about $7 billion and Involved, about 250,000 
man-years In student time.* _ A substantial portion of. this cost ik 
associated with the initial specialty training given to entrants to 
the enlisted force. Estimates are that in FY»19>6, over $2 billion 
ai^d 90,000 man-years of student time were devoted to initial voca- 
tional training. - ' 

Because of its large cost a great, deal of attention has been 
given to Improving the efficiency of initial spec laity -training . Re- 
search In this area lias focused almost exclusively on one aspect of 
training efficiency— technical efficiency. That is, researchers have 
tried to determine the mix of training inputs (curriculum choice, 
teaching aids, staff, etc.) that most ef f Iciently will achieve a cer- 
tain level o€ proficiency In graduates. 

The problem of selecting courses (and corresponding levels of 
graduate proficiency) Involves another aspect of training efficiency — 
economic efficiency. To evaluate specialty training in terms of 
economic efficiency ^ one must compare the benefits of additional for- 
mal training with the costs of obtaining those benef its . . This report 
describes early results of research that is designed to address the 

4 

question of economic efficiency in initial specialty training. ^ 

To evaluate the economic efficiency of training, one mast assess 
the on-the-job performance of graduates of alternative training pro- 
grams. Because additional formal training is always more costly, it 
is only worthwhile if it'results in suff icient** Improvements in on-(he- 
Job performance to offset the additional cost. The key to evaluating 
economic efficiency in training is therefore the development of re- 
liable measures of on-the-job performance. fhese measures need to 
reflect, net rather than gross productivity, to reflect productivity 



*E«tlmatea of training costs derived ff^om Mij^itarxj Manpower Train 
ing Report for FY 1976, Office of the Assistant Secretary of Defense 
(HR&A), March 1975. 
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oviM- t Inu» ratluM than at'a sln^^le point /and io bo baBinl on I ho pei - 
tonnant'o of nptu Illf Individuals rather lhan on the per lomianro ol 
a ropresental Ivo ind Iv Idiui 1 . 

Moasuros ^t hat ha^/o t hoso ^proport los ran bo lollottod by a variety 
ol mothods, Ini ludln^ dirort luoasuromonl , ) i>b--knowl odgo tost??>viUul 
siiporvlsory ratlu^sl Kai h o! t hoso mot hods has advantagos and dls- 
advantagos, and \\o ono approaih Is protoi'ablo, prinui taoio, to I ho 
^ ot hors. ' ^ ^ 

This roporl pre>vIdos a prollmlnary analysis ot a d^i^a sot that 
basN^HMi assoinblttd to oxploro tho(tradootf amonR training roursoH of 
ditforent longlhs. Tho data t^nslst of onl 1 stod suporv Isors ' oHtimatOB 
ot the not prodno t Iv 1 1 y of Iralnoos at dlfforont polntjL In their firBt 
torm ot military sorviro. rTieHo ost im^dji's aro used to cojistruct pro- 
fllos ol the timo path of ^iroduo t'lv* it )^ . A.miinher of productivity 
paths are presented to show tho gonoral rliaracter oC^tbe data we have 
CO 1 lor t od . • ' 

While our analysis Is admittedly slmpllstlrs the shaped -of and 
relat ionships .among the proi^ict ly ity profile's are quite conalst^nt 
with our prior expectat loi^ . Although further research Ib required 
before conclusions can be drawn regarding efficient mixes between 
formal and on-the-job training, the results of this 8tu<;ly suggest that 
It Is possible to draw meaningful conclusions from data based on the 
carefully constructed ratings of Supervisors. 
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I . INTRODUCTION^ 



I . 

Alnvost all new entr^^s Into the military attend a formal course 
of instruction in the skills required in their military occupational 
specialties. It is estimated that about' 320.000 of the 350,000 en- • 
listed personnel Joining the military in FY 1976 will receive such 
training. Since this init^ial specialty training will involve about 
80,000 man- years, of the trainees' time and cost about $2 bijlioh, 
the importance of conducting it efficiently i» obvious. Even small 
improvements in -efficiency can result in. substantial , recurring savings. 

. ■ A great deal of research has been devoted , to Impi^ovipg •"technical 
efficiency." To design technically efficient training courses, it is 
necesk^ry to select the least-cost mix of curriculum, instructional 
staff, teabhing aids, etc., that will produce graduates having a given 
•level of proficiency. There are,' however, an "infinite number of tech- 
nically efficient courses\in a given specialty, eath one of which will 
provide' its graduates with a different level of proficiency. The prob- 
lem of selecting ainong these courses, (ahd corresponding levels of pro- 
ficiency) involves another aspect of training ef f iciency--ecohoihic 
efficiency. Since almost any set of job skills aould be taught en- . 
tirelyon the Job, formal specialty training could be totally dis- 
-continued without losing the ability to maintain an effective military 
forc*.^ The reasons" for having formal specialty training are ^onomic 
rather than technological. To evaluate the optimal amount of formal, 
"training, one must compare the benefits of additional training with., 
its costs. The costs of formal training (faculty and studlht salaries, 
supplies, etc.) are obJious. The returns are less obvious because they 
take the form of impro>^ements in trainees' on-the-job performance. 
Economic efficiency in formal training is attaiped when the last dollar 



^Tbis report was presented at the Xand Conference on Defense Man- 
power, February 4-6, .1976. , - ^ 

^These estimates are derived" from Military Manpower Trai/niy H0- 
portfor FY. Office ' the Assistant Secretary' of Defense (Man- 

power and Reserve Affairs), March 1975. • y' 



spent on Increasing tUe amount" of formal tralnlnc Ini'f eases the bene- 

^ . y ^ ^ * . V . 

f its'^rec^lved after training by nnedolliar^ 

Rand Is currently conduc t log ^research to evaluate the^ cost- 
effectiveness of different alV^jJrlty of ^formal Initial specialty train- 
Ing. Slnc^ more'lnltlal tjafnllig Is always more ^ costly t4mn less/ 
longej courses arc? oirly cost-effective If they ptoduce better-qualified 
gr^dua€es.^ The key qaestlojv; tii ere fore , Is how can, the quality of 
frradu^l^^Q be Judged? "Op^ way 1^ to eompa^re the flrst--term productivity 
of graduates, of dlffeirient courses. - At 3 minimum, .longer courses must 
jna'ke graduates^ sut f ic tejit ly' ni(t)re productive during their first term to 

'compensate for the time spent In school If, for example, graduates 
of a C45urse that lasted an additional 2 months were no more productive 
tjjan graduates of a i?Viort^r course, the shoi^ter course would^learly 
Be preferable because, It^ graduates would be In the itillitary labor 
farce 2 months Ipnrger during, their enlistment. 

^ The. value of \ suiph productivity comparisons is, however, limited. 
Suppose, to.contitiuie the previous example, that graudates of the longer 

/course proved to b^ 5 percent more f)rorfuctlve after compl^eting the 
course than^r^duates of, the shorter one, even, after allowing for the 
fact that they had 2''raonths less on the job. How is one to Judge / 
whether the* ingrease in productivity is worth the extra *cost of the' 

loitger c'fearse? % 

' ' - 

j;^ To analyze' such tradeoffs, one must, be able to measure both the 
losts and^the benefits of^ longer courses in the same terms, and the 
conpndn ^denopiinator with the most general applicability is dollars. 
If ' the analysis is conducted in these terms, one can not only compare 
the efficiency of courses of different lengths, but can also consider 
other Margins' of substitution. For example, total •first-term train- 
ing co^ts can bla reduced by reducing the number of new entrants each 
year: — perhaps by substituting career personnel or capital goods for 
f ijr 8 t-r termers — as well a^ by choosing more efficient course lengths. 

We have a!)opted an approach to this problem that is an appllca- 
tion of contemporary human capit;al theory. Basic to this apprach is 
the notion that the costs of and returns to training can be measured 
by comparing an ^individiial 's pay and allowances with his net produc- 
tivity. In the representative case, the first term of service can 
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be thought of as having three distinct phases: first period of 
sic military training and initial technical training; second, a 



T>eriod of on-the-job training when the value of the individual s 

3 

productivity is less than the pay and allowances he, receives; and 
third, a period during which the military earns returns on Its In-, 
vestments In training because the person's productivity is greater 
than his pay and allowances. . These three periods are illustrated In 
Fig. 1, where the time t^ represents the end of formal training, the 
time t^ the end of the period of investment \x\ on- the-- job training, 
and t^^tne end of the first term of service. m 

During foAal training, the indiv idlial ' s direct contribution to 
militarV capability is "by definition zero. (Since he is not assigned 
to an cyperating unit, he could not possibly be '^contributing to current 
milita/y effectiveness.) His net contr ibutloh, hoVfeVer, is negative, 
because training requires resources (instructional staff, classroom 
spac^, etc.) that could otherwise be qo^itribut ing to current military 
capability. Therefore, the full cost of a person's formaT training, 
say at time t, includes both the pay and allowances he receives (g^iven 
by the dls^tance AB) and, the opportunity cost of' the other resources 
devoted to his training (BC). After he completes formal training, his, 
net contribution to military capability can be measured aa the differ- 
"^ence between his direct contribution and the opportunity cost of re- 
sources (such as the supervisors' time) devoted to his on-the-JplT 
training. The cost of on-the-job training at a point such as t can 
be rrfeasured by th^ difference between the individual's pay and -^Uow- 
ances'and his net productivity at that time (in this case the distance 
DE).^ The total cost of , on-the-job training (OJT) is approximately 
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i More precisely* this period Is one pf military investment , In on- 
the-job training, since on-the-job training in the sense of Improve-- 
ment in job-relevant skills continues as loAg as one's productivity 1« 
incr^sing. 

^In Fig* 1, the trainee's net productivity ie shown as being nega- 
tive immediately after he completes his formal training. Negative 
producti\rity would occur if th^ forgone prodactlvity of the talnee's 
supervisors exceeds the trainee's direct productivity. Although this 
is not necessarily the case, it does appear to be common. 



12 



i 5 
the shaded area In Fig. !• Finally, there are returns to training 

whenever th6 Individual's net productivity exceeda his pay and allow- 
ances. These^can be measured In the same fashion as the coats of on- 
the-JoT> training and are represented by the cross-hatched area in ' 
Fig. 1. • 

The difference betyefen the present value'^of the total* cost and 
the returns to training constitutes the military's net investment in 
an individual's first-term training. Naturally, the value of the net 
investment will change with clianges in the length of formal training.^ 
For example, longer courses will increase the cost of formal training. 
The returns fr6m thene Investments are in the form of decreases in 
the cost of on-the-job training and increases in the returns to train- 
ing resulting from enhanced on-the-job performance. The relative 
magnitudes of the two effects determine whether the change in the 
amount of training la cost-effective* 

In general, we expect the relationship between length of formal 
tr^ning and net investment in training to be like that shown in Fig. 2. 
This relationship is consistent with the view that, initially, ad- 
ditional amounts of foi^al training will reduce total training costs, 
but that at some point, the gains, in terms of enhanced on-the-job 
performance, will not outweigh the costs of the. add it lonal formal train- 
Ing and, hence^ net costs will not be reduced. In fact, at some point, 
it is reasonable to assume that additional amounts of formal training 
will cause net costs to rise. The potential number of such tradeoffs 
is, of course, quite large, and the current research program could not 
attempt to explore all the possible alternatives. Our approach to the 

problem ^1h to use existing natural experiments In training lengths, in 

- f 

< 4 * * 

y V- 

The sliaded area in Fig. 1 represents the undlscounted sum of on- 
the-job training costs. Because these costs arei Incurred at different 
liolnts, they should be measured as a discounted ^nm. Similarly, for- 
mal training costs and the returns to training shhAld be measured as 
present values, although, as a practical matter, formal training 
periods are typically so short that discounted and undlscounted values 
are virtually Identical. 

^Assuming that we are choosing from among the set of technically 
eff Ic lent courses. 




Length of formal training 

Fig. 2 — Relationship between length of formal tralr^^ng and net 
< Invettment In training 
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a variety of specialties to Raage the nature of (a) the potential 
economies to be derived from more efficient course length and (b) the 
factors that influence them. 

^The major limitation to putting this sort of model int^ operation 
Is' the difficulty of estimating the time path of productivity. ,We have 
developed a method in which Survey data are used to estimate produc- 
tivity, and we have recently f inishe^d collecting, data on members of 
about 50 specialties in the three ipajor se^Hfce branches. This data 
set consists of supervisory ratings ^f_Jjie^et contribution to unit 
production of selected first-term personnel serving at their first-duty 
station. These ratings, apply to each individual In the sample at a 
number of points during his first term of service. They thus permit 
Individual productivity profiles to be constructed for^ entire first 

t 



tormi T\\v ttnlinlqiio used ti> tlienu tat Ing^s way llmt of a h^o- 

qutMiftlulma 1 1 Hiirvoy. An Initial survey qnost lonna Ire was ma 1 1 c\l to 
sejeitod tirHt-^term personnel Identified by personnel rt^eords as en- 
.llstees servlnK t luM r flrst-dnty station. TJils Murvey was de- 
signed to verify Aample criteria and to Ident I f y. enl Istrd supervisors 
most faml 1 lar^ wltli the sAmpLed f Ir st -t ermer ' s work, A stibtlequenl sur- 
vey quest Umna Ire was mailed to Identified enlisted supervisors to 
elU It tlie prixluet iv I ty ratings previously described. In its final 
form, tlie productivity data set consists^of background material on 
Individual first-term personnel drawn frotn service per sonnei^records 
and survey material, a set of productivity ratings from an enlisted 
supervlt^or or supervisors, and additional background material on the 
supervisory raters tbemselves. This data set, tlien, forms the basis 
for tlie productivity analysis of the general model. 

Tlie purpose of this report Is to describe this data set and pome 
of the re{/ults of our early analysis of It. pi Section 11* the kinds 
of productivity measuiH^ment s we have, used are described in terms of 
the reasons for choosing them rather tlian as alterna-t IVes . In vSectlon 
ill, we present estimated prcxluct Iv i t y curves (or six military occupa- 
tlonal spec laities and compare them to Indicate the types of curves, ' 
and the relat lonsli Ips among them, tliat resu 1 t«^d^ f T*om our data. Sec- 
tion IV sunmiarlzes the findings of the study and indicates the need 
.. ■ • • ^ 

for and directions ol furtlier researcli In this area. 
1 
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II. PRODUCTIVITY MEASUREMENT 



The two moat difficult problems ^asoci-ated wl th adequately address- 
ing the ^ssue of ecoivJinlc efficiency in military specialty training 
are (1) to establish an appropriate analytic framework for comparing 
the benefits and costs of additloiTal amounts of training and, slnc^ 
the benefits of additional training take the form of increased produc- 
tivity. (2) to estimate productivity. We believe that the human capital 
model Just outlined provides an appropriate framework for analyzing the 
effects of different amounts «f formal training. In this section, we 
deal with the problem of productivity estimation. The objective is 
tolassess the advantages and disadvantages of several alternative 
api\oaches to the problem, focusing on^both their conceptual appropri- 
ateness and the cost of gathering the type of productivity data needed 
to address the issue of economic efficiency in specialty training. The 
first 'step in this process la to determine the properties that are Im- 
po^rtant in such measures^ *• 

PR OPERTIES OF PRODUCT IV IT? MEASUI^ES • . ^ 

Three properties are crucial in any. productivity measures used 
to evaluate alternative specialty training pollcl|s. The measures . 
should (1) permit productivity to be estimated over time, not Just at 
a single point, (2) measure net rather than gross productivity, and 
(3) be linked with the characteristics of the individual to whom they 
apply. First, it is^important that' the time path-of productivity be 
measured, because different types of formal training may affect pro- 
ductivity differently during the course of a military career. For 
example., if one were comparing two equally costly training courses in 
«hich one course emphasized the skills needed by a mature technician 
and the other emphasized^ the skills that an enlistee would use in the 
early months of his first-duty assignment, a n|p8ure that captured 
only productivity in the early months after completion of training 
would favor the latter course^ whereas a measure that focused on pro- 
ductivity later in the career would favor the former^ Adequate evalu- 
ation requites comparisons of productivity at a number of points so 



/ 
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that comparisons can be made among^. courses with difffrap^tal effects 

I\ on productivity ove*vtlme. , " , ' £ 

/ Second, It Is llnpprtant that net ^a.ther than gross productivity 
■be measured. An Individual's gross productivity la the amount Ke 
. personally pi^oduces; his net productivity Is the difference between \ 
♦ / the unit's production In his presence, and In hls^absence . The two ^ 

need not b^ at «ll the same, and the relationship between them c^n be 
expected to change systematically with experience. Consider for 
example, a new, specialty school graduate who Joins a. larg^ radio 
, repair shop. Although he will probably be able to complete some sim- 

ple types *f repairs, he will almost certainly need fairly close 
supervlsloij. His gross productivity in these circumstances is posl- 
tlve, but if the reduction in his supervisors' production exceeds the 
^ trainee's direct production, his i\^t productivity is negative. As he 

acquires more experience, his gross pr^uctivity will normally in- 
crease, and the amount of supervision he requires will decrease, so 
that net productivity will rise more rapidly than gross productivity. 
At some point, supervision vfiXl become minimal, and gross and net pi;fi- 
ductivlty will, for all intents and purposes, be identical. To the 
extent that the two measures differ", net productivity Is clearly 'the 
appropriate concept for evaluating training because it measures <he 
difference in military effectiveness attributable to an individual. 
The use of gross productivity tends to bias analyses of substitutions 
between 'formal and on-the-job training in favor of on-thew^Job training 
by overstating the trainee's productivity durin|^ his early experience 
when supervisory inputs- are greatest. 

The ability to relate productivity measures \nd trainee character- 
istics, such as aptitude, education, attitudes, etc., is essential to 
a detailed analysis of training alternatives. OtSherwise, it is ex- 
tremely difficult to control for personal character istics in assessing 
the effects of alternative amounts of formal training. Lacking these 
relationships, one is essentially restricted' to making comparisons 



^Of course, for those with substantial supervisory responsibilities, 
the relatlonahip is reversed and net productivity exceeds gross produc-- 
tivity. This •circumstance Is not likely te be relevant during the 
period of interest here, 
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among reprosentat hrr fud iv IdualH or to performing' yJ^P^r ijnentH Involv- 



/ 

Ing* matched HampleH. The former approach 1b undoHlrable because one 
can never be certain how representative the "reprenentat Ive indl* 
vlduaT* iH. The lattej approach Is uodeslrable because It prechidVB > 
the use \if natural experiments and because the experiment In this 
ttase (Inducting formal aAd on-TlW-Job training) Is quite lengthy, 

• 

Both approaches are lln\ited In the -^aense that the relationships that 
are estimated are conditional on a given mix of trainee attributes — 
tliey provide no information on the effects of changes In trainee 
attributes. , - 



ALlEJRNA'r IVK^^^M^^^ 

Measures having the properties \\\Bt discussed cqn be gathered In 
a number of ways. Here |/»fe describe the strengths and weaknesses of 
two of the major alternatives, which provides a basis for evaluating 
the nieasurement procedure we have used. ^ , 

^Subst 1 tut Ion Measurenient 

The general character of the productivity measures one would col- 
lect with unlimited resources Is fairly straightforward/ Such measures 

yould Involve estimating an Individual's net productivity by measuring 
a unit's output In his presence and in his absence. In our example of 
the radio repair shop, tl>e shop's output couhl be meafAired with its 
full romplemeiYt of personnel ancl^ then with various combinations of 

,n -1 personnel • The d if fertince , between the output of repairs with 
and without a given person Is a measure of his net product Ivl ty • A 
time path of net productivity could^e estimated either cross-sect ion- 
ally or longitudinally. ^ ^ , 

One substantial difficulty in ImpiWentlng this approach is the 
measurement of unit output. The problem arises prlmatlly because there 
are many different typ^s of output In a given specialty. To return to 
the radio shop example: Within a given shop, several ^ypes of radios 
will be malntiilned and many types of failures will occyr. If <here are 
substantial varlatsions in the difficulty of repairing different types 
of "failures and in the mix of failure,8^ver time, the numbe^ of "re- 
paired radios" that can be fumed out in. a given number of man-hours 

^19 
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ylll dlffcf considerably. To. take these variations into account, 
weights must be developed for different types of repairs, and output 
must he measured as a Weighted sum of repairs. When t-he context Is 
broadened to Incliuie mulrtole shops, the development of appropriate 
welghtH is *ven more impol;tant, because differences in equipment mixes 
Anong shops can Introduces substantial differenfces In measured produc-, 
tlvlty if inappropriate weights are selected. , ♦ . 

At first glance It may aeem that once an unambiguous definition 
of output i» developed, a small number of observations will be suf- 
ficient to evaluate two alternative training strategies because the 
productivity measurements can presumably be made quite precisely. 
This Is not true, hoS*ver, because of the large number of factors 
besides military training that influence a person's^ contribution to 
unit performance. These Include (a) his motivation, ability, previous 
education,, etc. ; (b) the number, experience level, motivation, ability, 
and previous education of other personnel in the unit; (c) the group's 
experience In working together; (d) the stock of capital equipment; 
and (e) the demand for service that the shop faces. To control for 
these factors, a large number of observations are necessary. 

It should be apparent from the preceding discussion that the 
"ideal" sort of productivity measures wowid be quite costly to assemble 
and would. In spite of the cost, be less than perfect. This suggests 
that it is worthwhile to consider other alternatives. If suitable 
less costly alternatives can be found, they" would be preferable if 
only because of the limited amount of previous research in fbis area. 
Since the information available is so limited, it is probably better 
to obtain a fdrst approximation of productivity and economic e/ficiency 
of training for- a number of specjialties than to devote the saineV^mount 
of resources to a detailed analysis of one or two specialities. 

Job-Performance Tests * 

Ohe approach that retains the character iSt ic of direct measure- 
ment but involves measures that are simpler to develop and administer 
is the use of Job-performance tests. This approach Involves testing 
individuals on a specific set of skills used in their specialty. By 
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testing people with different amounts of on-the'-Job experience, /or one 
person at several tlm«(8, the relationship between pi^odut Li v it y and 
experience Van be established; and^y linking these vieaeurements to 
those of formal technical training, background, ability, etc., one 
can estimate the effects of different initial training (controlling 
for periional characteristics), ' ^ / 

There are several major limitations to this approach, however. 
Most Importantly, it Involves measurement of gross, rather than net, 
productivity. The on-the-job training process in the military gen- 
erally Involves substantial inputs of supervisory resources, and there- 
fore differences between net and gross productivity are likely to be 
large, especially early in the on-the-job training process. Use of 
gross productivity will result in downward-biased estimates of on-the- 
job training costs and, therefore (because' the returns to formal train- 
ing are under stated) , will lead to policies that entail l^ss-than- 
efficient amounts of forn^l training. A more subtle bias occurs be- 
cause of differentials in supervision across individuals. One would ex-^ 
pect the level of. supervisory inputs to the on-the-job training process 
to vary with the trainee's amount of r formal training and his personal 
attributes; If, for example, better tralgaed, more able personnel re- 
quire Ifess supervision, grQss productivity measures understate the 
differential in performance between the bet ter-trainecL, more able 
individuals and those with less ability and training. This implies 
that gross productivity measures will fail to capture part of the re- 
t-urns to additional formal draining and also th^t the relationships 
between personal characteristics and productivity estimated from gross 
productivity data will be biased. Fina]Lly, there are real questions 
of how well Job-performance tests measure actual gross productivity. 
Even if the chosen Job tasks accurately reflect the duties in a par- 
ticular specialty, they may not^accurately represent a' particular in- 
dividual's attual dutie^. Further, Job tests measure a person's oapa-^ 
bility to perform those, tasks, not his actual performance of them. . 
The difference in the observed performance of two similar persons with, 
for .example, different levels of illotlvation is likely to be much smaller 
dui:ing a short test tl^an over the course of several days or weeks on 
the Job. 



Rand Method^ • » 



The apf^olich we have choaen useH aupervlHora' ratings of net 
productivity rather than direct measurement:. Thin approach has the 
d Isadva^itage of being a subject Ive^measur^, but It also has many ad- 
vantages, eapeclally since there has been • 89 little previous analysis ^ 
of the reialClonshlp between formal specialty training and productivity. 
In the following discussion, we will describe the productivity measure- 
ments that have been gathered and examine the strong and weak points 
of the approach. 

We have used a self-admlnlstered mall questionnaire to obtain 

8 

supervllbors' ratings of the net productivity o^ specific trainees at 
several points during their* flrit term of service. Specifically^ 
supervisors we!^e asked to rate each trainee's productivity (1) durltig 
hl6 first month with the unit, (2) at the time that the rating was 
completed, ^J) 1 year from the time of rating^ and (A) after completion 
of 4 years of service. In each Instance, the supervisor was asked to' 
rate the Individual's net produdt Ivlty ralatlve to that of the typical 
specialist with 4 years of experience. Together, these points trace 
out an estimated . time' path of relative net productivity that can be 
related to the attributes of the person being rated. 

This method of estimating productivity Is attractive for four 
reasons. First, It measui^es net rather than gross produci: Ivlty . 
Second, It doe's permit one to control for personal characteristics In 
comparing training alternat Ivea. Third, the cost of data collection 
Is relatively low, which means that for a given budget miiny more u 
specialties can be analyzed than would be possible using, for example, 
substitution measures. Although substitution measures would presumably 
lead to more precise estimates^ it appears more valuable, at the present 
stage of research in this area, to explore the general m^g^ltude amd 
pattern of training effects across a number of specialties than to 
analyze a small number of specialties in great detail. Fourth, be- 
cause of the general nature of the measures that are obtained, compar- 
isons across specialties are feasible both within a given service and 



Q 

Net productivity is defined as the difference between a trainee's 
groas productivity and the forgone productivity of supervisors who 



work with hltti. 
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across services. This fcidture Is also Important In (;^btalnlng a broad 

^^'^ overview of the effects of training on productivity. ^ 

< • • " >^ 

* ^ Two important limitations to our approach should be considered. 

« 

Flf^t, the concept of net productivity Is fairly sophisticated ^nd ^ 
it Is not one that enlisted supervl^^ots are likely to have be€j^ famil- 
iar with before receiving the questionnaire. Thus^lt Is possible that 
some survey responses will be Invalid because supervisors did not under- 
stand what they were being asked to. do. Because of this possibility, 
we spent a great deal of tlme^and effort In thfe survey design stage 
to develop and field test « clear explanation of the concept < of net 
productivity. Of course, no such explanat lOn^ould be clear to all 
rec|.plents of the questionnaire, so a simple test of comprehension was' 
Included In the survey Instrument. In addition to their questionnaire 
responses, s|ipervlsors w^sre asked to rat^ both a "typical" technical 
school graduate and a directed duty assignment trainee;^ this rating 

was Intended to provide some Insight Into the supervisor's comprehen- 

J 

slon of the concept. Clearly, some responses will still be unusable 
because of the rater's Inability to comprehend the conQept of net pro- 
ductivity, but a preliminary analysis of the data does not indicate 
that this is a serious problem. 

Another potential limitation is the possibjLllty of Important dif- 
— ferences among raters in their rating systems — some tend to rate easy 

and. others hard; some tend to see people as very similar and some as 
very dissimilar. These differences have the poterittal of producing 
substantial noise in the data, and a pilot study of our approach 
suggested that they were indeed a serious consideration. An econo- 
metric model for dealing Vith this source of distortion by estimating 
parameters of the supervisor's rating system was developed In conjunc- ^ 
t Ion with the pilot study, and has been expanded and tested through^ 
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A directed duty assignment trainee Is one who goes directly from 
basic military training -to an operating unit without attending a for- 
mal specialty training school. * 

M. Gay, Estimating the Cost of On-the-Job Training in Military 
Oocupations : A Methodology and Pilot Study ^ The Rand Corporation, 
R-1351-ARPA, April 1974, 
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11 ^ 
Mon^e' Carlo simulatiot^/ ^ It provides, a means for controlling for 

supervisor-crating effects. The ratings of a j^ypical trainee provide 

another method of determining distortfon, since one would expect, for 

example, the supervjrfbr^^i^o is a harder-than-average rater to give both 

the "typical tra/nee" and individual trainees low ratings. A pre- 

llminary analysis of our data suggests that a good deal of the noise 

in the supervisor ratings is. In fact, eliminated by controlling for 

the. supervisor *s rating of the typical trainee. 



V Richard V.L. Cooper and Gary R. Nelson, Analytic Methods for 
Adjustiruf Subjective Ratincj Schemes y The Rand Corpdration, R-1685- 
ARPA, June 1976. . 
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in. PRODUCTIVITY RELATIONSHIPS 



The data hase Ys^embled at Rand to provide a vehicle f or analyzing 
^the effects of> different amounts of flrs^T-term specialty training is 
very large' and, in many respects, unique. It includes information on 
individuals in more than 50 .specialt ies In Whe three major service 
branches. When preliminary data processing is completed, an observa- 
tion will include (a) the supervl^sor ' s rating of a trainee, (b) back- 
' ground Information from service personnel records on both the trainee 
and the supervisor, and • (c) additional background information on the 
trainee obtained from a survey he had completed. The survey data were 
collected throug^li sequential mail surveys of about 30,000 first-term 
enlisted personnel and an approximately equal number of enlisted super- 
visors. This section presents the findings of a very preliminary 
analysis of these data. Because we are at a very early stage in our 
work, we have adopted a rather elementary form of analysis— compari- 
sons of estimat^ed productivity functions over time in several special- 
ties. In presenting these results, we inteod to suggest a broad con- 
sistency between the observed patterns of productivity and those one 
would have expected a priori. '» 

This section is divided Into three parts. First, we describe the 
data used in the present analysis. Second, we present a Qomparison 
of estimated productivity curves for six military occupational special 
ties. Third, by analyzing estimated product iv-ify curves for the "typ- 
leal" technical school graduate and directed dj^ty assignment traj-nee, 
' we illustrate both the potent^l for and the pitfalls of using ihese . 
sorts of data to analyze effects of training on productivity. 

■ , ' \ 

DATA CHARACTERISTICS 

As mentioned previously, our productivity data consist of re- 
sponses to questionnaires administered by mall to supervisors in a 
selected set of military occupational specialties. Supervisors were 
requested t^^ovide three types of estimates of net productivity over 



time. First, and most impprtant, we asked for estimates of specific 



Ind^KTlduals' net contribution to unit prodqctlvity at faur points 

during their first term of aervlce: (1) during the first month on 

the Job, (2) at the time that the rating was completed (which will, 

of course. Imply different amounts of on-the-job experience for dlf- 

ferent individuals), (3) 1 year from the time of %he rating, (^nd (A) 

after* A years ot tiervlce. Second, we asked for estimates of the typ^ 

» 

leal technical school graduate's net contribution to unit production 

t 

during his first month on the Job, after 1 year on the Job, after 2 
years on the job, and after A years of service. Third, we aaked for jf 
estimates of the typical directed duty assignment tt^alnee's net con-^ 
trlbutlon to unit production after 1 month, 1 year, and 2 years on 
the Job and after) A years of Service. 

For purposes of the present analysis, we have selected d sub- 
sample of 6 of the more than 50 military occupational speclal|/*e^f or 
which data have been collected. Th^^ subsaH^ple covers a range of job'^ 
tasks and Includes a set of comparable specialties in different serv- 
ices. We have selected the light weapons Infantry specialists (llB) 
and the hospital corpsmen (91B) in the^Army; radio repair specialists 
(ETN) and hospital corpsmen (HM) in the Navy; radio repair specialists 
(30AXA) and hospital cof^smen (902X0) in the Air Force. Observations 
for an" individual wer^^ncluded only if a complete 8^t;of four produc- ^ 
tlvlty estimates was available, and if the trainee (a) vwbs serving at 
his first-duty station; (b) had attended technical school training, 
and (3) ^as serving in the specialty in which he was trained A Further , 
to eliminate responses clearly indicating that the supervisor ' ^com- 
prehension of the concepts was poor, we deleted cases in which tne 
supervisor rated the typical technical school graduate's [productivity 

jit 100 percent or more during his first month on the JOir^or at zero 

12 

or less after A years of service. 

COMPARISON OF PRODUCTIVITY CURVES FOR SIX SPECIALTIES 

Representative productivity curves for members of a glyen spf^cialty 
can4>e estimated by using ratings of the product ivij^jt-^dx specif Ic 
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Using these criteria, the number of cases in each specialty were 
IIB, 96; 91B, 197; ETN, 252; ItM, 85; 304X4, 689; and 902X0, 363. ^ 



individuals, as lHu8trat«d for two Navy occupational specialties 

(radio repalmen and hospital corpsmen) In Fig. 3. Average produc- . 

tlvlty during the flr^t month on the Job and after 4 years of service 

(points 1 .and 8 In Fig. 3) can be estimated by taking the average value 

13 ' . 

of tW responses for all Individuals lo the samp^.^ Intermediate 

points are estimated by t/iklng the average estimate of productivity at 
the present time ai\^ 1 year from now for groups pf Individuals In dif- 
ferent experience categories. Specifically, we have grouped Individuals 
Into three experience categories: ,5-9, 10-14, and 15-19 mont^is of Job 
experience. Points 2-4 In Fig. 3 represent ttie mean values of the 
estimates of productivity at the time of the rating for those In each 
of the categories (plotted at the midpoints of the Intervals) . Points 
5-7 represent the average values of estimated productivity a year from 
the time of the rating for people in the same experience categories. 

On average, these estimates of relative productivity" for both the 
✓* radio repairmen and htispdWl corpsmen conform to our expectations. 

The curves have positive slopes, reflecting an increase In productivity 
as a function of experience. Further, estimated rel^ative productivity 
after 4 years of experience approaches 100 percent in both specialties. 
Since productivity is being measured relative to the average specialist 
with 4 years of experience, average productivity at this point would, 
in the absence of bias, be 100 percent. The averaged ratings of the 
radio repairmen are consistently below those for the hopsital corps- 
men, which is to be expected because radio repair is more technically 
demanding and requires more experience before one can attain proficiency 
FlRure 4 shows similar comparisons for the Air Force hospital corpsmen 
and radio repairmen, whose productivity patterns are similar to those 
of the Navy specialists. Initial net productivity is negative for radio 
repairmen and positive for hospital corpsmen, and the differences in 



•^^■'since the horizontal axis of Fig. 3 measurea on-the-job expe- . 
rlence and' point 8 corresponds to. productivity after 4 years of aervioa, 
the position of point 8 in a given specialty depends on the length of 
fornal technical training ift that, specialty. 

^^Cells with fewer than 10 observations were not included in the 
plots. . 

^^This at least partly explains why the difference between the two 
curves dininishes over time. 

27 
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rclatlve productivity diminish with experience. The curves for the 
tw6 Army specialties (hospltU corpsmen and light weapons infantry- 
nen) shown in Fig. 5 are less clear-cut. Here it Is not so obvious 
which Is the more technically demanding specialty; moreover, the dif- 
ferential in productivity does not decline over time ai^ expected. 
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Another way of exploring the consistency of our estimates is by 
comparing estimates of similar specialties In different services. The 
two curves in Fig. 6 show the productivity prof ilesv in two cadi o re- 
pair specialties. Navy ETN and Air Force AFSC 304X4. Here a striking 
similarity can be observed In both ini'?tal^ and- final productivity and 
rate of progress. The curves suggest that there is a high degree of 
con^arability between the two services in this occupational specialty. 

Figure 7 makes a similar comparison of the productivity ratings 
for hospital corpsmen In the three services. Again the consonance 
among the curves is striking In terms of both, slopes and intetcepts. 
However, the avera^f productivity curve for the Air Force corpsmen has 
a slightljif^loweir Inffedal value than the curves ^or the Army and Navy 
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cofprnien. One possible explanation Is that hospital, corpsmen In the 
Air Force receive approximately 20 percent less formal schooling than 
thdaeMn the Army and Navy, but this hypothesis cannot be tcJsted In 
the current analytic framework* " ^ 

S#rvlcevlde productivity ciirves for both hospital corpsmen and 
radio repairmen are presented In Fig- 8. The relationships In this 



Instance are quite similar to those 
in Figs. 3 and 4. 



observed for the Navy' and Air Force 
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As a group, these results are quite encouraging. They are con-^ 
slstent ^th expectations both In terms of the productivity patterns 
observed over time In a given specialty and the relationships among 
specialties. These findings sugjgest that the productivity estimates 
we have assembled will be useful In appraising alternative training 
policies. 



^^These curves Include all members of the respective specialties 
regardless' of service. ^ ^ 
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COMPARISON OF PRODUCTIVITY CURyES FOR G RAPUAT tlS AND NOWGRAP UATES 

Both the .|Jotentl«l for and the pitfalls of using theae types of 
data to ^analyEe* the ef facta of training on productivity can be lllus- 
trate(| by some comparlaoha of productivity curves estimated as aver- 
ages of supervisors' ratings of the "typical" technical school gradu- 
ate's performance and the performance of the '^typical" directed duty 
assignment trainee. Unlike the curves estimated from ratings of spe- 
cific Individuals, these curves chart the progress of hypothetical ( 
typical trainees through the first term of service. 

The four curves in Fig. 9 present such comparisons for two radio 
repair specialties: Navy ETN and Air Force 30AxV. Thle curves exhibit 
properties consistent with the general ^pectatlons of productivity 
estimates and are similar to the average curves estimated for specific 
Individuals* The curves for the Navy> radio repairmen (represented by 
the solid lines) sho<^ substantial and persistent differences betveen 




Flg.9— Comparlicn of combined producHvlty profllA of "fyplcal"^aduate 
and nohgroduatm rrainaes In two radio rapair cpecialHei : >y 
/ Navy ETN> and Air Fore. 304X4 
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th€ performance of graduaC^B and nongriiduatea throughout the first 
4 yeiva of service -end suggest that returns to training may be sub- 
stantial. The curyes for the Air Force radio repairmen (represented 
by the dotted lines) show a similar relationship between the perfor- 
mance of graduates and nongraduatea, but the differences appear to 
be much smaller than tho^e between the two Navy curves. ^ 

Bas^d on these two sets ot curves alone, one might argue that 
there is a difference in training effectiveness between the Navy and 
Air Force., with higher returns tp training occurring in the Navy. 
Consideration of the two sets of curves together , however, raises 

V 

questions about tt^is interpretation. The two curves for the graduate 
repairmen are practically Identical; the differences between the * 
services arise because Navy nongraduates .do not perform as well as 
Ail Force nongraduat es. This suggests ^hat the apparent difference 
in returns to training between the two services may, in fact, be due 
to differences in the relative quality of graduates and nongraduates 
rather than to differences in training effectiveness. That is, Navy^ , 
and Air Force tec"hnical/ school graduates may be roughly comparable in 
terms of aptitude, elducation, etc.*, but Navy directed duty assignment 
trainees may be le^ss able than their Air Force counterparts. Of course, 
with this sort of analysis one cannot really sort out the effects of 
differences in personal characteristics from d if.f erences' in training 
effectiveness. The results do imply, however ,* t\iat ye are likely to 
observe systematic differences in t^e performance of graduates and 
nongraduates, and that in analyzing them it will be important to con^ 
trol for the personal characteristics of trainees. 
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IV. CONCLUSIONS 

To determine the efficient amount of technical school training 
for military occupational specialties, the analyses must be conducted 
within a framework that permits the analyst to estimate the effects 
o9 that training on pbs-ttralning Job performance. The approach we 
have chosen is an application of human capital thedry in which the 
costs of on-the-job training and the returns to training are measure^ 
by comparing the trainee* s pay and net productivity. The key element 
in implementing this approach is the estimation of bn-t he- Job produc- 
tivity. To construct estimates of the time path of product ivity, >fe 
have used supervisors* estimates of the net productivity of trainees 
at various points in their first term of service. A number of esti- 
mated ^productivity paths were preseVited to show the generjal character 
of the data we have collected. * 

All the ptoductlvity curves for tjie six specialties discussed in 

t 

this report showed a positive slope that tended to decrease j3ver time 
(the rate of improvement declined with experience). Moreover, com- 
parisons among specialties tended to cpnform ^ expectations, both 
when those comparisons were made between specialties in the same, 
^prvice and wHen they were made between Comparable specialties across 
services^ . 

Encouraging as these results are, they are of no direct help in 
solving the problem of efflc:i^ent amounts of technical school train- 
ing. Two steps are necessary before this problem can be addressed. 
First, the productivity estimates must be integrated into a broader 
framework so that training costs and returns to training can be esti- 
mated. Second, d«timated net training costs (formal training c'oats 
plus on-the-job tfalmlng costs less returns to training) must be 
analyzed for individuals with differing amounts of formal training. 
And, as the results ^ this study of ^'typical trainee" data suggest, 
future analyses must be done in a multivariate framework that controls 
for other factors that Influence estimated productivity « 



